HOW   TO  EVALUATE 
PACKAGED  SOFTWARE 


NOVEMBER  1981 


HOW  TO  EVALUATE  PACKAGED  SOFTWARE 


CONTENTS 


00C01 1 


REASONS  FOR  USING  PACKAGED  SOFTWARE 

A.  Lack  Of  Skilled  Personnel 

B.  User  Demands 

C.  Software  Cost  Savings 

D.  Lower  Risks 


SOURCES  OF  PACKAGED  SOFTWARE 

A.  Software  Companies 

B.  Software  Brokers 

C.  User  Groups 

D.  Service  Companies 

E.  Turnkey  Systems  Companies 

F.  Computer  Stores 

G.  Mainframe  Manufacturers 


III      EVALUATING  SOFTWARE  AND  SOFTWARE  VENDORS 

A.  Defining  Requirements 

B.  Locating  The  Software  Packages 

C.  Procedures  For  Requesting  Information 

D.  Vendor  Proposals  And  Presentations 

E.  Rating  The  Software 

1.  Functionality 

2.  Hardware  Requirements 

3.  Languages  And  Systems  Software 

4.  Performance 

5.  Flexibility 

6.  Installation 

7.  Documentation 

8.  Product  Support 

9.  Operational  Status 
10.      User  Groups 

I  I .  Origins 

12.  Value 

13.  Cost 

F.  Rating  The  Vendors 

1.  Experience 

2.  Size 

3.  Customer  Base 

4.  Profitability 


Page 

I 
I 

3 
4 
5 

7 
7 
8 
8 
9 
10 
II 
I  I 

13 
13 
14 

16 
18 
19 
19 
20 
20 
21 
21 
22 
22 
23 
24 
25 
25 
25 
26 
27 
27 
27 
27 
28 


i  - 


©1981  by  INPUT.  Reproduction  Prohibited. 


INPUT 


Pa2e 

5.  Primary  Markets  28 

6.  Service  Organization  29 

7.  References  29 

IV      CONTRACTS  AND  NEGOTIATIONS   31 

A.  Purchase  And  Lease  Terms  3 1 

B.  Installation  32 

C.  Maintenance  And  Enhancements  33 

D.  Modifications  34 

E.  Financial  Liability  34 

F.  Title  34 

G.  Other  Factors  35 


-  ii  - 

©1981  by  INPUT.  Reproduction  Prohibited.  INPUT 


HOW  TO  EVALUATE  PACKAGED  SOFTWARE 
EXHIBITS 

Page 

I      -I      The  Shortage  Of  Software  Personnel  2 


-  iii  - 

©1981  by  INPUT.  Reproduction  Prohibited. 


INPUT 


REASONS  FOR   USING  PACKAGED  SOFTWARE 


REASONS  FOR  USING  PACKAGED  SOFTWARE 


A.       LACK  OF  SKILLED  PERSONNEL 


•  The  substantial  reduction  in  the  cost  of  computing  power  in  the  past  decade 
has  spawned  an  extraordinary  proliferation  of  computer  systems.  Unfortu- 
nately for  the  owners  and  managers  of  these  systems,  the  population  of  skilled 
analysts  and  programmers  has  not  kept  pace  with  the  systems. 

•  There  is  a  critical  shortage  of  skilled  personnel  today,  and  it  has  been 
projected  that  that  shortage  will  increase  in  the  1980s,  as  illustrated  in  Exhibit 
l-l. 

•  The  cost  of  hardware  is  expected  to  continue  to  decline  in  the  1980s,  while  the 
cost  of  skilled  programmers  is  expected  to  continue  to  rise. 

•  Regardless  of  cost,  many  EDP  managers  are  going  to  find  that  skilled  systems 
personnel  are  just  not  available.  In  a  recent  INPUT  study,  60%  of  the 
respondents  reported  a  "moderate"  to  "severe"  shortage  of  systems  personnel. 

•  In  light  of  this  crisis,  packaged  software  may  present  the  only  possible  solution 
to  this  problem. 
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EXHIBIT  1-1 
THE  SHORTAGE  OF  SOFTWARE  PERSONNEL 
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USER  DEMANDS 


It  is  not  uncommon  today  for  data  processing  operations  to  have  backlogs  of 
user  applications  waiting  to  be  programmed. 

In  some  cases  the  backlog  is  measured  in  years,  not  months. 

Users  are  turning  increasingly  to  outside  service  organizations  to  fulfill 
their  needs. 

Users  are  demanding  immediate  solutions,  and  they  are  looking  for  alterna- 
tives when  their  in-house  data  processing  staff  says  "next  year." 

INPUT  has  reported  and  continues  to  forecast  an  explosive  growth  in  the  sale 
of  turnkey  systems.  This  growth  is  fueled  by: 

Unprecedented  user  demands. 

A  package  solution  made  possible  by  packaged  software. 

Usually,  the  systems  department  is  involved  in,  and  can  exercise  some  control 
over,  the  purchase  of  turnkey  systems;  but  that  is  rarely  the  case  in  the 
procurement  of  remote  computer  services. 

Many  corporations  have  users  who  spend  tens  of  thousands  of  dollars  a  month 
on  computer  services,  of  which  the  chief  data  processing  executive  is  not  even 
aware. 

Users  have  often  been  so  frustrated  in  their  attempts  to  fulfill  their 
system  requirements  that  they  no  longer  even  consider  the  in-house 
system  as  a  possible  alternative,  but  go  directly  to  an  outside  service 
organization. 
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•  The  kinds  of  solutions  that  the  user  is  seeking  and  finding  in  computer  services 
and  turnkey  systems  are  also  available  in  packaged  software. 

Packaged  software  is  available  when  the  user  needs  it,  which  is  now. 

•  It  is  also  available  at  a  reasonable  cost  -  often  a  fraction  of  the  cost  of  the 
alternatives. 

•  It  is  the  systems  department's  responsibility  to  see  that  the  user  considers  all 
the  alternatives  and  makes  a  wise  choice  among  them. 


C.       SOFTWARE  COST  SAVINGS 


•  Properly  selected  and  utilized  packaged  software  can  produce  enormous  cost 
savings  for  any  organization. 

Typically,  packaged  software  costs  less  than  10%  of  what  it  would  cost 
to  produce  in-house. 

Similar  savings  are  also  available  for  maintenance  and  enhancements  of 
the  software. 

•  Development  budgets  for  sophisticated  management  information  systems  (MIS) 
today  are  measured  in  millions  of  dollars. 

Only  the  largest  organizations  can  contemplate  such  projects,  and  only 
if  they  have  the  skilled  personnel  to  implement  them. 

A  number  of  these  systems  are  available  as  packaged  software  for  well 
under  $100,000  for  the  main  module. 
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LOWER  RISKS 


The  risks  involved  in  purchasing  packaged  software  are  substantially  lower 
than  the  risks  associated  with  the  development  of  software. 

Buying  packaged  software  involves  a  known  quantity.  The  package  has  been 
specified,  designed,  written,  documented,  tested,  de-bugged,  and  installed  and 
is  completely  operational. 

Building  software  involves  budgets  and  schedules  that  are  rarely  met. 

Building  software  also  often  requires  systems  or  applications  knowledge  which 
is  either  not  readily  available  or,  at  best,  somewhat  uncertain  and  unknown. 

Highly  specialized  hardware,  communications,  and  data  base  expertise  may  be 
required  for  applications  software  design.  These  skills  may  not  be  available  at 
an  affordable  price. 

Buying  packaged  software  is  not  a  reckless  process  if  one  follows  a  sound 
evaluation  process. 

It  is  the  purpose  of  this  paper  to  present  such  an  evaluation  process. 
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II 


SOURCES  OF  PACKAGED  SOFTWARE 


A.       SOFTWARE  COMPANIES 

•  There  are  over  1,000  firms  dedicated  to  the  development  and  sale  of  packaged 
software. 

These  firms  range  in  size  from  one-person  operations  to  thousands  of 
employees. 

Many  of  these  firms  specialize  in  one  software  product  while  others 
offer  a  rather  extensive  selection. 

The  average  software  house  offers  fewer  than  a  half  dozen  products. 

•  Software  companies  constitute  one  of  the  fastest  growth  industries  in  the 
United  States.  INPUT  estimates  that  the  software  industry  will  grow  at  33% 
for  the  next  five  years. 

•  Software  companies  generated  $3.6  billion  in  revenues  in  1981,  which  are 
projected  to  grow  to  $15  billion  by  1986. 

•  Most  software  companies  are  very  specialized,  sometimes  dealing  with  a 
specific  application  for  a  single  industry. 
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Many  software  companies  are  packaging  their  software  with  hardware  to 
produce  turnkey  systems. 


B.       SOFTWARE  BROKERS 


•  Most  software  companies  are  small,  single-location  firms  with  limited  market- 
ing capabilities. 

•  Software  brokers  lend  valuable  assistance  to  the  small  software  company  in 
the  sale  and  distribution  of  software. 

•  Some  software  brokers  are  fully  staffed  to  install  and  support  the  products 
they  sell  and  to  train  the  user's  personnel. 

•  The  quality  of  products  sold  by  brokers  can  vary  widely,  so  prospective 
purchasers  of  software  should  carefully  review  their  offerings. 


C.       USER  GROUPS 


•  Computer  user  groups  can  provide  sources  of  software  from  among  their 
membership. 

•  This  source  can  be  particularly  valuable  for  extremely  specialized  applications 
software,  since  software  companies  may  have  little  to  offer  to  such  a  small 
market. 

•  This  source  can  also  be  fraught  with  above-average  risks  in  that  the  software 
has  usually  been  developed  for  a  single  specific  user  and  the  needs  of  other 
users  may  not  have  been  considered  in  its  design. 
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•  The  quality  of  documentation  and  level  of  support  provided  for  software 
developed  by  users  are  generally  poorer  than  those  for  products  produced  by 
software  houses. 

•  These  packages  often  have  very  few  users  and  consequently  have  not  really 
undergone  the  tests  of  the  marketplace. 

•  Software  packages  developed  by  users  are  usually  inexpensive  and  may  be 
worth  their  cost.  However,  low-cost  software  can  be  difficult  to  use,  difficult 
to  install,  have  operational  "bugs,"  etc.  Extremely  careful  analysis  of  this 
software  package  alternative  is  required. 


D.       SERVICE  COMPANIES 


•  Remote  computing  services  (RCS)  companies  are  relatively  new  in  the 
packaged  software  business. 

•  Many  RCS  vendors  are  selling  their  software  as  a  part  of  either  a  turnkey 
system  or  a  user  site  hardware  service  (USHS). 

•  Some  of  their  software  is  available  in  an  unbundled  package,  either: 

Directly  from  the  RCS  company. 

From  a  third  party  who  originally  developed  the  software  and  licensed 
it  for  resale  to  the  RCS. 

•  Very  large  users  of  RCS  will  often  demand  -  as  a  condition  for  awarding  their 
business  to  a  vendor  -  the  right  to  purchase  the  software  in  the  future  for  use 
on  their  in-house  machines. 
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•  Buying  a  software  package  from  an  RCS  vendor  offers  the  unique  advantage  of 
being  able  to  use  the  software  on  a  trial  basis  in  the  service  environment  prior 
to  making  the  purchase  decision. 

•  RCS  vendors  are  also  generally  very  strong  in  the  areas  of  documentation, 
training,  and  support. 

•  Systems  departments  should  monitor  all  expenditures  for  RCS  and  acquaint 
themselves  with  any  purchase  options  available  on  the  software  being  used  or 
proposed. 

•  It  is  not  unusual  today  for  a  user  department  to  find  that  its  RCS  bill 
eventually  reaches  the  $50,000-a-month-or-higher  level  for  a  single-decision 
support  system. 

In  the  early  stages,  it  is  wise  to  plan  for  future  growth. 

Providing  for  a  purchase  option  with  an  RCS  vendor  can  result  in 
substantial  savings  as  usage  increases. 

The  other  alternative,  USHS,  can  also  provide  similar  benefits. 


TURNKEY  SYSTEMS  COMPANIES 


•  The  turnkey  systems  industry  is  growing  at  incredible  rates.   INPUT  estimates 
that  this  industry  grew  by  40%  from  I  979  to  I  980. 

•  These  firms  provide  all  the  hardware  and  software  as  a  package  to  satisfy  the 
needs  of  a  particular  application. 

•  The  growth  of  this  industry  indicates  that  users  like  vendors  who  assume 
complete  responsibility  for  their  applications. 
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•  Turnkey  systems  are  available  in  a  range  of  prices  from  under  $1,000  to  over 
$1  million. 

•  Turnkey  systems   should   be  explored  particularly  for  industry-specialized 
applications. 


F.       COMPUTER  STORES 


•  Computer  stores  are  becoming  very  valuable  sources  of  packaged  software. 

•  The  stores  often  act  as  brokers  or  distributors  of  software  vendors. 

•  A  number  of  the  personal  computer  manufacturers  are  developing  and  selling 
serious  software  for  the  business  market. 

•  Systems  managers  should  visit  their  neighborhood  computer  store. 

•  Some  of  the  information  analysis  packaged  software  available  for  personal 
computers  is  surprisingly  powerful  and  very  suitable  to  business  planning 
applications. 

G.       MAINFRAME  MANUFACTURERS 


•  Computer  manufacturers  should  be  considered  when  evaluating  packaged 
software. 

•  Users  should  not  consider  "free"  software  free  because  the  cost  of  installing 
and  using  a  "free"  package  can  exceed  the  cost  of  a  purchased  one. 
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•  Computer  manufacturers  of  software  may  require  upgrading  a  machine  by 
adding  peripherals  which  might  not  be  required  with  another  vendor's  packaged 
software. 

•  Users  should  subject  all  packaged  software,  whether  supplied  by  their  equip- 
ment manufacturer  or  not,  to  a  very  careful  evaluation. 

•  Users  should  not  limit  their  search  for  packaged  software  to  the  manufac- 
turers of  their  equipment.  Independent  software  companies  often  provide 
software  that  is  superior  to  the  manufacturer's  package. 


Manufacturers  are  generally  not  very  highly  motivated  to  sell  the  most 
efficient  software. 

Most  major  computer  manufacturers  in  the  U.S.  have  recently  established 
programs  whereby  they  acquire  the  right  to  sell  software  developed  by 
independent  software  vendors.  Users  benefit  from  the  expertise  of  the 
independent  vendor,  and  from  the  support  and  endorsement  of  the  manu- 
facturer. 
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EVALUATING   SOFTWARE   AND  SOFTWARE 

VENDORS 


Ill       EVALUATING  SOFTWARE  AND  SOFTWARE  VENDORS 


A.       DEFINING  REQUIREMENTS 

•  As  with  a  package  one  would  develop  in-house,  the  first  step  -  and  the  most 
important  -  is  to  define  the  requirements  for  the  software  package. 

•  EDP  managers  should  develop  fairly  detailed  specifications  of  the  job  to  be 
performed  by  the  software  including: 

All  data  to  be  input. 

Calculations  to  be  performed. 

Output  to  be  produced. 

Volumes  of  inputs  and  outputs. 

Communications  required. 

Interface  with  existing  and  planned  systems  and  procedures. 
Any  requirements  in  respect  to  equipment  to  be  used. 
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Any  requirements  for  specific  operating  systems  or  programming 
languages. 

Time  constraints  for  input,  processing,  and/or  output. 

•  Although  users  may  prefer  to  run  the  software  on  their  existing  hardware  and 
operating  system,  and  with  a  specific  language,  they  should  evaluate  all 
software  without  those  restrictions. 

The  best  packaged  software  to  fulfill  the  requirements  may  not  be 
available  in  that  specific  environment. 

The  right  software  may  require  procuring  new  or  additional  equipment 
to  run  it  on. 

•  All  prospective  users  of  the  software  should  be  involved  in,  and  approve,  the 
software  specifications. 

•  In  defining  the  requirements,  one  should  be  liberal  in  the  assumptions  about 
the  growth  of  the  applications  software  usage,  and  plan  for  at  least  five  years 
of  growth. 

•  Categorize  each  requirement  as  either  essential  or  desirable. 

•  The  specifications  should  be  for  the  ideal  system,  but  one  should  be  prepared 
to  compromise  or  change  requirements. 


B.       LOCATING  THE  SOFTWARE  PACKAGES 


There  are  over  10,000  software  packages  on  the  market. 
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It  is  very  likely  that  someone  has  already  written  a  package  that  will 
fulfill  most  of  the  requirements  users  have  defined. 

With  a  little  effort,  users  should  be  able  to  find  a  number  of  packages 
for  their  consideration. 

Three  of  the  best  sources  for  listing  likely  packages  are  International 
Computer  Program  (1CP),  Datapro,  and  Auerbach  publications. 

User  groups  can  often  provide  information  about  available  software. 

Trade  associations  can  often  provide  software  information. 

Trade  publications  such  as  Datamation,  Infosystems,  and  Interface  often 
publish  lists  of  available  software  as  well  as  advertisements  for  packaged 
software. 

Trade  shows  often  provide  a  forum  for  software  vendors  who  cater  to  a 
specific  industry. 

Advertisements  in  the  trade  press  also  call  attention  to  available  products. 
The  products  advertised  are  generally  from  the  large  vendors  and  may  not  be 
the  best  package  to  solve  a  user's  problem. 

Specialized  consulting  firms  such  as  INPUT  can  provide  assistance  in  identify- 
ing packages  and  vendors. 

Users  within  the  same  industry  may  be  able  to  assist  in  identifying  software 
packages. 
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PROCEDURES  FOR  REQUESTING  INFORMATION 


•  A  prospective  purchaser  of  a  software  package  should  realize  that  the 
evaluation  process  can  be  very  time-consuming. 

If  the  software  package  is  very  important  to  the  organization  and/or 
very  expensive,  one  should  allocate  sufficient  time  to  evaluate  the 
products. 

Procedures  can  be  established  which  will  minimize  the  time  required  to 
evaluate  products  and  vendors. 

•  One  procedure  that  should  be  used  is  the  request  for  information  (RFI). 

The  request  for  information  is  a  tool  used  for  the  preliminary  screening 
of  candidate  software  packages  that  enables  the  buyer  to  systematic- 
ally review  the  prospective  packages  with  a  minimal  expenditure  of 
time. 

The  RFI  should  spell  out  the  ground  rules  of  the  evaluation  process  and 
the  information  required  to  complete  that  process. 

Defined  requirements  should  be  compiled  into  two  checklists: 

The  first  checklist  should  contain  all  requirements  that  are 
essential  and  critical.  This  list  should  contain  only  items  that 
are  mandatory,  as  it  will  be  used  in  the  initial  elimination  round. 

The  second  checklist  should  contain  all  the  desirable  require- 
ments. These  items  should  be  rated  according  to  their  desir- 
ability, but  the  ratings  should  not  be  disclosed  to  the  vendors. 
The  ratings  can  be  used  to  score  the  packages  in  the  second 
elimination  round. 
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In  addition  to  the  checklist,  prospective  buyers  should  define  the 
general  environment  in  which  they  would  like  to  operate  the  software, 
which  should  include  a  description  of  the  system,  its  peripherals, 
operating  system,  and  language. 

The  above  list  should  be  put  into  a  letter  and  sent  to  all  the  selected 
prospective  vendors.  The  letter  should  point  out  that  only  written 
responses  are  acceptable  and  that  sales  calls  are  not  welcome. 

Buyers  should  give  the  name  of  one  individual  in  their  organiza- 
tion who  will  respond  to  phone  inquiries  to  provide  clarifying 
information  in  respect  to  the  RFI. 

The  vendor  should  be  told  to  send  appropriate  product  literature, 
price  lists,  contracts,  and  general  information  about  the  com- 
pany. If  the  company  does  not  have  an  annual  report,  some  type 
of  profile  should  be  included.  A  general  capabilities  brochure  or 
profile  describing  all  the  company's  products  would  also  be 
desirable. 

Prospective  buyers  should  make  it  clear  in  their  letter  that  they  are  not 
requesting  a  proposal  at  this  time,  but  only  general  information,  and  an 
indication  of  whether  the  vendor  has  the  capabilities  required  in  its 
software  package. 

They  should  also  indicate  that  the  material  is  required  by  a 
certain  date,  which  should  be  about  three  weeks  from  when  the 
letter  is  sent. 

Prospective  buyers  should  tell  each  vendor  that  proposals  or 
further  information  will  be  solicited  from  the  most  promising 
candidates  within  a  defined  timeframe.  One  month  from  the 
above  due  date  should  provide  adequate  time  for  the  initial 
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screening.  All  vendors  should  be  notified  of  a  decision  by  the 
date  given. 


D.       VENDOR  PROPOSALS  AND  PRESENTATIONS 


•  After  all  the  vendors  have  responded  to  the  RFI,  the  information  gathered 
should  be  evaluated  and  all  but  two  to  five  vendors  eliminated. 

•  At  this  point,  one  should  issue  a  request  for  proposals  (RFP). 

•  The  RFP  should  be  more  specific  than  the  RFI  and  may,  in  fact,  be  expanded 
based  on  the  information  initially  provided  by  the  vendors. 

•  An  extensive  list  of  evaluation  factors  is  provided  in  the  following  two 
sections  (E  and  F). 

A  number  of  these  can  be  used  in  the  RFI  stage. 

All  of  them  should  be  addressed  when  soliciting  and  evaluating  pro- 
posals. 

•  In  addition  to  submitting  proposals,  vendors  should  be  invited  to  make  on-site 
presentations  to  the  selection  committee. 

•  In  both  the  proposal  and  the  presentations,  vendors  should  be  directed  to 
address  specific  issues  that  are  particularly  important. 

•  The  presentation  is  especially  valuable  in  evaluating  the  people  with  whom  one 
will  be  doing  business. 
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If  the  software  one  is  buying  is  a  significant  investment  and  will  require 
continued  interface  with  the  vendor's  marketing  and  technical  people, 
this  is  an  excellent  time  to  assess  them. 

Remember:  it  may  be  impressive  when  the  vendor's  senior  management 
and  headquarters  specialists  make  the  presentation,  but  it  will  be  the 
local  representatives  with  whom  one  will  have  the  most  contact  and  on 
whom  one  will  mostly  depend  for  service. 

•  After  all  the  proposals  and  presentations  are  made,  one  should  evaluate  all  the 
data  and  make  a  decision. 

•  Once  the  decision  is  made,  one  should  negotiate  the  terms  of  the  agreement. 
Advice  and  information  on  contracts  and  negotiations  are  offered  in  Chapter 
IV. 

E.       RATING  THE  SOFTWARE 

I.  FUNCTIONALITY 

•  The  software  should  meet  all  the  minimum  specifications. 

If  none  of  the  vendors  was  able  to  answer  all  the  minimum  specifica- 
tions, those  specifications  should  be  reviewed  to  determine  if  they  are 
really  essential. 

If  the  requirements  are  indeed  valid,  but  no  vendor  could  satisfy  all  of 
them,  a  prospective  buyer  may  have  to  broaden  the  search  or  request 
that  the  most  promising  vendors  propose  how  they  could  comply  with 
the  minimum  specifications. 
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•  If  a  number  of  vendors  were  able  to  satisfy  the  minimal  requirements,  the  next 
step  is  to  evaluate  how  well  they  satisfy  the  desirable  requirements. 

Some  vendors  may  have  added  desirable  items  that  were  not  on  the 
checklist.  These  items  should  also  be  assigned  a  rating  among  the  other 
desirable  requirements. 

The  ratings  should  be  added  up  for  all  the  desirables  checked  on  the  list, 
and  a  total  score  derived  for  each  vendor. 

The  candidate  vendors  should  now  be  narrowed  down  to  three  to  six 
vendors. 

2.  HARDWARE  REQUIREMENTS 

•  One  should  determine  whether  the  software  package  will  operate  in  the 
specified  hardware  environment. 

•  If  it  does,  there  is  no  problem. 

•  If  it  does  not,  one  should  determine  the  cost  of  additional  hardware  and  factor 
that  into  the  decision. 

The  software  that  is  most  consistent  with  the  requirements  might  be 
available  only  on  a  turnkey  system,  which  should  be  considered  on  a 
total  cost  basis. 

3.  LANGUAGES  AND  SYSTEMS  SOFTWARE 

•  Prospective  buyers  should  determine  whether: 

The  software  is  compatible  with  the  operating  system  utilized  in  their 
installation. 


-  20- 

©1981  by  INPUT.  Reproduction  Prohibited. 


INPUT 


The  language  employed  is  familiar  to  their  programmers. 

It  will  be  necessary  for  their  programmers  to  support,  modify,  or 
enhance  the  software  in  the  future. 

The  package  is  written  in  a  standard  language  that  is  widely  available 
and  supported. 

The  software  is  available  to  run  in  a  wide  range  of  environments  and 
system  configurations,  whether  there  are  trade-offs  in  performance 
and/or  cost  in  the  different  environments. 

PERFORMANCE 

How  efficiently  the  software  performs  may  be  an  important  criterion  in  the 
selection  process.  Prospective  buyers  should: 

Ask  the  vendor  to  demonstrate  the  software  for  them. 

Run  benchmarks  and  compare  software  performance. 

Solicit  performance  information  from  other  users  of  the  software. 

If  the  software  cannot  be  demonstrated  or  is  not  operational,  then  that  should 
weigh  heavily  against  the  vendor  involved. 

FLEXIBILITY 

Certain  products  are  extremely  flexible,  which  may  be  a  plus  in  prospective 
buyers'  estimations  of  that  product,  but  they  should  be  aware  that  the  added 
flexibility  usually  means  added  cost.  If  they  don't  need  all  the  flexibility,  they 
may  be  paying  for  more  than  they  can  use. 
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•  In  addition  to  front-end  cost,  flexibility  usually  impacts  performance  by 
reducing  efficiency  of  execution  and/or  requiring  extraordinary  machine 
resources. 

6.  INSTALLATION 

•  The  vendor  should  be  prepared  to  provide  a  level  of  support  commensurate 
with  the  cost  and  complexity  of  the  package  to  be  installed. 

•  The  vendor  should  provide  a  clear  outline  of  what  is  required  to  install  the 
package,  which  should  include: 

Any  considerations  involving  a  conversion  from  the  buyer's  current 
system  to  the  new  installation. 

Tra  ining  required  for  all  personnel,  from  clerks  to  programmers  to 
managers  and  users. 

Qualifications  and  background  of  the  vendor's  personnel  who  will  be 
providing  installation  support  and  training. 

A  reasonable  time  schedule  for  the  implementation  of  the  software. 

7.  DOCUMENTATION 

•  Clear,  comprehensive,  understandable  documentation  is  essential  for  the 
effective  use  of  packaged  software. 

•  Appropriate  documentation  should  be  available  for  all  personnel  who  interface 
with  the  system,  including: 

Clerks. 

Analysts. 
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Programmers. 


Operators. 
Users. 

The  vendor  will  not  be  as  readily  available  to  answer  questions  regarding  the 
software  as  in-house  analysts  and  programmers,  so  the  standards  and  quality  of 
the  documentation  should  be  very  high. 

The  care  taken  to  document  packaged  software  will  probably  reflect  the  care 
taken  in  its  design  and  development,  and  so  it  may  also  be  a  measure  of  the 
quality  of  the  software  itself. 

PRODUCT  SUPPORT 

The  level  of  continuing  product  support  that  the  vendor  provides  should  be  an 
important  criterion  in  software  selection. 

It  should  be  determined  whether  the  vendor  has  a  good  distribution  system  for 
both  software  fixes  and  releases  of  system  enhancement. 

Prospective  buyers  should  determine  whether: 

The  software  can  be  impacted  by  exogenous  factors  such  as  changes  in 
the  law  (taxes,  for  example)  and  if  the  vendor  has  a  history  of 
responding  in  a  timely  fashion  to  such  changes. 

The  product  is  important  enough  to  the  vendor  so  that  he  is  continu- 
ously upgrading  it  and  releasing  new  versions. 

The  releases  have  a  minimal  impact  on  installations. 
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The  releases  involve  additional  fees,  or  whether  they  are  covered  in  the 
basic  agreement. 

Training  is  provided,  both  for  initial  and  ongoing  requirements. 

Training  is  provided  on  a  bundled  (no  additional  cost)  or  unbundled 
(additional  cost)  basis,  and  how  that  may  change  in  the  future. 

Need  for  on-site  assistance  and  consultation  is  anticipated,  and  whether 
the  vendor  can  provide  it. 

9.        OPERATIONAL  STATUS 

•  Software  vendors  have  been  known  to  indicate  that  their  software  has  certain 
capabilities  which  may  not  in  fact  be  operational. 

The  vendor  may  have  sincere  intentions  to  provide  these  capabilities 
prior  to  installation,  but  they  may  not  be  as  reliable  and  proven  as 
expected. 

A  check  with  other  users,  as  well  as  demonstrations  and  benchmarks, 
should  prove  that  all  capabilities  are  there  as  claimed. 

•  One  of  the  reasons  packaged  software  is  an  attractive  alternative  to  in-house 
development  is  that  most  of  the  risks  of  development  have  been  eliminated. 

If  the  software  is  not  fully  operational,  buyers  will  be  assuming  some 
risk  they  originally  intended  to  avoid. 

If  development  must  be  conducted  to  bring  the  software  up  to  require- 
ments, its  cost  and  schedule  should  be  clearly  spelled  out  in  the 
contract,  along  with  appropriate  penalties  for  failure  to  perform. 
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USER  GROUPS 


Certain  software  packages  are  so  widely  used  (a  few  with  over  1,000 
installations)  that  user  groups  have  been  formed  by  the  vendors'  customers. 

Prospective  buyers  should  determine  whether  such  a  group  exists  and  consider 
it  a  real  plus  if  it  does. 

User  groups  not  only  can  provide  valuable  assistance  in  the  selection  process, 
but  also  can  exercise  strong  leverage  with  the  vendor  when  enhancements  are 
being  designed. 

ORIGINS 

It  is  worthwhile  to  explore  the  genesis  of  a  packaged  software  product. 

A  product  that  was  developed  with  the  intent  of  selling  it  to  a  broad  market  is 
generally  much  better  than  a  product  that  was  developed  for  a  single  end  user 
(then  modified  or  enhanced  for  resale  to  other  installations). 

VALUE 

Buyers  should  assess  the  value  of  the  packaged  software  in  benefits  measured 
in  dollars. 

Buyers  should  estimate  how  much  it  is  worth  to  solve  the  problem  which  the 
software  addresses. 

If  a  dollar  value  can  be  assigned  to  the  various  requirements  specified,  more 
scientific  evaluation  criteria  can  be  established  for  reviewing  the  alternatives 
offered  by  the  various  vendors. 
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13.  COST 

•  All  the  cost,  both  direct  and  indirect,  should  be  estimated  for  acquiring  and 
using  each  software  product  evaluated. 

•  Cost  can  vary  substantially  from  vendor  to  vendor  for  products  that  appear 
essentially  the  same.  This  can  be  due  to  differences  in  pricing  schedules  for 
various  options  or  the  differing  amounts  of  support  provided  for  the  packages 
by  the  vendors. 

•  Items  that  will  contribute  to  the  total  cost  of  a  software  package  include: 

The  price  of  the  software. 
Maintenance  fees. 
Modification  costs. 
Enhancement  costs. 
Additional  hardware  costs. 
System  and  procedure  changes. 
File  conversions. 
Training  costs. 

Additional  personnel  required. 
Installation  costs. 
Production  operation. 
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The  cost  should  not  only  be  less  than  the  value  received,  but  should  also  De 
substantially  lower  than  the  in-house  development  cost  for  the  same  system. 


RATING  THE  VENDORS 


1.  EXPERIENCE 

•  The  experience  a  vendor  has  should  be  evaluated  in  several  ways,  including: 

The  amount  of  time  a  vendor  has  been  in  the  software  business. 

The  amount  of  time  a  vendor  has  sold  the  particular  package. 

The  experience  of  the  top  executives  of  the  vendor  prior  to  entering  the 
software  business. 

2.  SIZE 

•  Generally,  larger  organizations  should  rank  higher  than  smaller  ones. 

•  If  an  organization  has  more  than  $1  million  in  annual  revenues  or  more  than  20 
employees,  it  is  probably  satisfactory. 

3.  CUSTOMER  BASE 

•  A  large  customer  base  is  definitely  desirable  in  a  vendor. 

•  It  is  particularly  good  when  a  vendor  can  provide  a  list  of  reference  customers 
using  the  software  product. 
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•  The  number  of  customers  should  be  evaluated  relative  to  the  total  number  of 
potential  users.  A  highly  specialized  industry  package  will  have  a  much 
smaller  market  than  a  general  data  base  management  system. 

4.  PROFITABILITY 

•  Vendors  with  demonstrated  financial  stability  should  be  preferred. 

•  The  continued  useful  life  of  a  software  product  is  dependent  upon  the  vendor's 
ability  to  continuously  enhance  and  maintain  it,  which  to  some  extent  is 
dependent  upon  a  positive  cash  flow. 

•  If  the  vendor  is  not  a  public  company  (most  are  not)  and  if  the  software 
package  requires  a  substantial  financial  commitment,  then  one  should  request 
copies  of  audited  financial  statements  going  back  as  far  as  five  years. 

5.  PRIMARY  MARKETS 

•  Vendors  should  be  evaluated  on  the  basis  of  their  commitment  to  the  software 
business. 

•  Buyers  should  give  higher  ratings  to  a  vendor  that  is  primarily  dependent  on 
software  sales,  as  opposed  to  a  vendor  for  which  software  is  only  a  secondary 
line  of  business. 

•  In  addition,  buyers  should  look  more  favorably  upon  a  software  product  if  it  is 
the  premier  or  principal  product  of  the  company,  rather  than  just  one  among  a 
stable  of  products. 

•  If  the  product  is  an  industry  application  specialty  product,  buyers  should 
compare  the  level  of  commitment  each  vendor  has  toward  that  industry 
segment. 
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SERVICE  ORGANIZATION 


If  the  software  package  is  going  to  be  used  at  one  location,  buyers  should 
determine  the  proximity  of  the  vendor's  nearest  office. 

If  the  package  is  going  to  be  employed  in  multiple  locations,  buyers  should 
determine  whether  the  vendor  can  provide  support  at  each  location. 

Regardless  of  geographical  proximity,  buyers  should  rate  the  level  of  compe- 
tence that  the  vendor  can  provide  in  support  of  the  package. 

Generally,  a  staff  (even  one  person)  dedicated  to  the  support  of  a  single 
product  is  better  than  one  individual  supporting  a  variety  of  software 
products. 

A  "dedicated"  headquarters  support  staff  is  also  important,  as  long  as  it 
is  accessible. 

REFERENCES 

It  is  very  important  that  references  from  users  of  the  package  be  requested 
and  randomly  checked. 

If  possible,  prospective  buyers  should  visit  an  installation  similar  to  the  one  for 
which  they  are  buying  the  software. 

They  should  compile  a  list  of  at  least  five  questions  to  ask  other  users 
regarding  key  concerns  they  have  about  the  software  package  and  the  vendor. 

References  should  be  requested  and  checked  only  when  buyers  are  about  to 
make  a  final  decision.  They  should  be  used  primarily  to  verify  and  validate  the 
results  of  the  buyers'  own  investigations. 
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CONTRACTS   AND  NEGOTIATIONS 


IV       CONTRACTS  AND  NEGOTIATIONS 


A.       PURCHASE  AND  LEASE  TERMS 


•  When  packaged  software  is  purchased,  a  buyer  is  usually  paying  for  a  perpetual 
license  for  use  of  the  product.  Perpetual  licenses  are  employed  by  sellers  of 
software  products  for  several  reasons: 

This  form  of  "sale"  protects  the  vendor's  title  to  the  product. 

The  license  also  allows  the  vendor  to  control  and  restrict  the  future 
sale  of  the  product  by  the  licensee  to  others. 

•  If  prospective  buyers  have  any  intention  to  resell  the  packaged  software,  they 
should  purchase  the  product,  including  title  to  it. 

•  Leases  are  usually  offered  for  a  software  product  for  a  minimum  of  one  or  two 
years. 

The  typical  fee  for  a  two-year  lease  is  in  the  range  of  3%  to  5%  of  the 
purchase  price  per  month. 

Leases  for  less  than  two  years  and  month-to-month  rentals,  when 
available,  will  usually  be  at  a  higher  rate  and  will  often  involve 
installation  charges. 
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Acquiring  packaged  software  is  most  often  a  long-term  commitment  on  the 
buyer's  part,  who  should  consider  a  "purchase"  rather  than  a  lease  if: 

The  funds  are  available. 

The  buyer's  cost  of  funds  is  lower  than  the  vendor's,  which  is  often  the 
case. 

Some  vendors  will  consider  extending  the  purchase  payments  over  a  90-day 
period  to  help  buyers  who  have  budget  or  other  constraints  on  their  purchasing 
authority. 

If  a  purchase  credit  is  not  offered  against  a  part  of  lease  payments,  buyers 
should  request  its  inclusion  in  the  lease  contract. 


•  Although  the  vendor  may  have  a  standard  agreement,  buyers  should  remember 
that  many  parts  of  a  contract  are  negotiable. 

•  When  comparing  package  costs  offered  by  several  vendors  with  different 
prices  and  terms: 

Establish  a  common  denominator,  such  as  the  expected  useful  life  of 
the  package  or  term  of  the  lease. 

Compute  the  total  cost  of  using  each  vendor's  package  over  that 
common  period. 


B.  INSTALLATION 


Normally  a  certain  amount  of  the  installation  cost  is  covered  by  the  purchase 
price. 
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•  Buyers  should  be  sure  that  the  vendor's  responsibilities  for  assisting  in  the 
installation  are  written  in  detail  in  the  contract. 

•  Penalties  should  be  detailed  in  the  contract  for  incomplete  or  late  installation 
of  the  product  when  the  vendor  is  responsible. 

•  The  amount  and  kind  of  installation  support  and  training  should  be  defined  in 
the  contract. 

It  should  include  a  description  of  the  qualifications  of  the  personnel  who 
will  provide  the  on-site  support,  or  even  the  names  of  the  specific 
individuals. 


C.       MAINTENANCE  AND  ENHANCEMENTS 


After  installation,  a  period  for  evaluation  should  be  set  aside  before  final 
acceptance  of  the  package. 

The  period  should  cover  one  full  cycle  of  use  of  the  package. 

If  the  package  is  found  to  be  not  acceptable,  buyers  should  be  able  to 
return  it  and  obtain  a  full  refund. 

The  package  should  be  warranted  to  be  free  of  bugs  for  at  least  one  year. 

Procedures  for  correcting  bugs  after  installation  should  be  provided. 

A  minimum  response  time  for  finding  bugs  should  also  be  defined. 

Many  vendors  offer  maintenance  agreements  that  cover  enhancements  in 
addition  to  repairs. 
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Buyers  should  try  to  get  the  right  to  buy  new  versions  of  the  package 
minus  the  price  they  have  already  paid  for  the  original  version. 

Maintenance  agreements  are  requisite  when  frequent  technological  or 
exogenous  changes  can  be  expected  to  impact  the  usefulness  of  the 
software. 


D.  MODIFICATIONS 


•  Any  modifications  the  vendor  has  agreed  to  should  be  included  in  the  contract 
in  detail,  with  due  dates. 

•  If  buyers   intend   to   modify  the  package,  they  should  be  sure  that  the 
modifications  do  not  invalidate  any  warranties  or  maintenance  agreements. 


E.       FINANCIAL  LIABILITY 


The  vendor  should  assume  financial  liability  at  least  up  to  the  purchase  price 
of  the  package  if  it  fails  to  perform  as  warranted  and  specified. 


TITLE 


Vendors  should  guarantee  that  the  product  is  proprietary,  and  that  buyers  will 
be  defended  against  any  infringement  claims  involving  trademarks,  trade 
secrets,  or  patents. 
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•  Vendors  will  expect  buyers  to  protect  the  vendor's  trade  secrets  as  if  those 
trade  secrets  were  the  buyer's  own,  and  not  to  reproduce  the  materials  the 
vendor  supplied. 

G.       OTHER  FACTORS 

•  The  contract  should  incorporate,  as  referenced  attachments,  the  proposal  and 
all  relevant  sales  literature. 

•  If  the  buyer  intends  to  use  the  package  at  several  locations  or  on  several 
machines,  that  right  should  be  included  in  the  contract. 

The  vendor  will  probably  expect  additional  fees  for  multiple  use,  but 
these  should  be  at  a  discount  from  the  standard  fees. 

The  buyer  should  also  consider  the  potential  use  of  the  software  by 
subsidiaries  or  other  affiliates. 

•  The  vendor  company  may  request  the  right  to  assign  any  lease  payments  to  a 
third  organization,  but  it  should  not  be  allowed  to  assign  its  obligations  under 
the  agreement  without  the  buyer's  approval. 

•  The  contract  should  define  what  the  vendor  is  going  to  deliver  to  the  buyer, 
including: 

Decks  of  cards. 

Tapes. 

Source  documents. 
Documentation. 
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User  guides. 
Training. 
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